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SYSTEMS  SAFTTY 


Systems  Safety  has  been  called  a  number  of  things  over  the  years.  It 
has  been  called  Design  Safety;  lt*s  been  called  Engineering  Safety,  Planning 
or  Management  Safety.  Systems  Safety  encompasses  all  of  those.  A  system  is 
considered  as  all  the  equipment,  all  the  actions,  all  the  parts  necessary 
for  operation.  For  example,  the  F-16  system  does  not  just  refer  to  X  number 
of  airplanes  and  pilots,  but  the  whole  system;  with  crew  training,  the 
syllabus  of  instruction  for  all  the  maintenance  personnel,  the  software 
tapes  that  go  into  producing  the  automatic  test  equipment,  the  Seek  Eagle  or 
Stores  Qualification  program  conducted  here  at  the  Center  to  qualify  and 
determine  the  delivery  procedures  for  all  the  external  stores  and  weapons 
carried  on  the  airplane,  etc. 

Now  consider  for  a  moment  your  role  as  a  test  engineer  and  a  test  pilot 
and  the  overall  design  in  testing  of  an  airplane  or  a  weapon  system.  You 
are  the  last  person  in  the  design  effort  to  review  the  safety  of  that 
machine  or  device  before  it's  produced  and  delivered  to  your  compatriots  out 
in  the  field.  It's  up  to  you  to  identify  the  faults  and  hazards  of  those 
systems  and  get  them  corrected  before  they  are  turned  loose  to  the 
operational  field. 

Consider  for  a  moment  some  of  the  equipment  that  you've  been  flying 
around  the  Air  Force  over  the  past  years  and  the  hazard  associated  with 
those  systems.  Maybe  you've  wondered  why  those  problems  exist,  and  why  they 
weren't  discovered  and  corrected  while  the  airplane  was  being  tested.  For 
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exaiipl.,  thOM  of  jra,  *0  hm*  n<w  On  F-4  «Ul  r«!.ll  ttat  >4»n  you  put 
your  guar  <loun,  ««  oftoo  tl*.  not,  you'U  lone  your  lACgll.  Ifj  .  design 
fault  in  the  airplane  and  partl«fl«-iy  tPe  looatlon  of  the  MCM  antenna  on 
the  noaegear  door.  Mian  the  idisals  are  doun,  the  gear  doors  timi  90  degrees 
and  now  the  anten»  1.  perallel  to  the  fuselage  of  the  airplane  fre,p»ntly 
««Blng  the  nCM  to  break  lock,  u  that  dsalai  f«iit  an  oversight  on  the 
part  of  the  test  pilot  and  the  teat  engineers  Mio  oonduct  the  original 

develop-nt  test  on  the  F-t7  ihe  en««r  Is  ■!»■.  Deed  the  final  t«*ni,el 
report  on  the  F-k  series  of  elroreft!  you  «U1  find  the  oonoluelon  that  the 
aCAk  breaks  look,  with  the  gear  do*  and  the  reoonendatlon  that  this  be 

fixed.  It  was  never  done  and  you  aOght  ask  uhy.  OiaracterlstlosUy,  the 

reports  that  you  spend  so  »oh  thaa  and  effort  preparli,  are  never  real  by 
the  Prograa  Kanager,  or  at  least  not  In  tli»  to  sot  M>on  your 
reo<,.,andatlo„s.  Houevar,  there  are  tuo  tools  at  o«-  dlsposel,  through  the 
Systaio  Safety  ,roo«»,  that  «U1  assist  wi  In  oorraitlng  these  kinds  of 
*slgn  faults.  First,  In  the  ovaraU  systea.  safety  pooeasi  the  3P0  or  the 
Prograa  Manager  teroes  the  oontraotor  to  design  safety  Into  his  systea,  to 
revleu  any  faults  or  any  hazards  dlsoovered  during  the  testli*  and  to  alt 
*>""  ulth  you,  the  tester,  the  prcgr-  aanager,  the  u«r,  or  «d  the 
loglstlos  coamnd  to  discuss  these  probleas  parlodloally.  IherelUre 
probleas  like  this  are  recognized  at  the  tlae  that  they  occur,  rather  than 
soae  tlae  later  idw  the  final  report  Is  prepared.  Che  other  tool  to 
promote  Identlfloatlon  of  design  failts.  Is  a  neu  Section  V  of  Ieoh  Order 
00-35D-5k.  Ibis  tech  order  deal,  -rtth  uhat  Is  kno«  as  service  reports, 
previously  called  deficiency  reports.  (See  Attaohaent  1  for  details.)  As 


3-H 


testor  you  cite  the  deficiency  shortly  after  discovery  and  identify  it  to 
the  progran  manager,  who  may  close  it  out  without  action  or  may  put  up  the 
money  and  give  direction  to  the  contractor  to  correct  the  problon.  This  is 
a  very  powerful  tool  for  correcting  flaws  similar  to  the  F-4  TACAN  exanple. 
Recently  in  that  a  new  flight  director  was  to  be  Incorporated  on  the 
airplane.  The  night  director  had  a  series  of  faults,  some  of  v*lch  were 
downright  hazardous.  Because  of  service  reports  and  prompt  reaction  by  the 
SPO  to  systems  safety  working  group  discussions,  fixes  were  developed  and 
implemented  prior  to  the  final  test  report  submission.  The  report,  having 
been  in  typing  for  so  long,  concluded  that  the  flight  director  did  not  work 
correctly  and  recommended  that  it  be  fixed.  By  prompt  action  by  the  test 
pilot  and  test  engineer,  this  design  fault  was  corrected  before  the  final 
report  was  ever  published! 

It's  Important  to  understand  the  role  of  safety  in  management.  Too 
01  ten  you  think  of  the  role  of  a  safety  staff  as  being  a  cop  who  goes  out 
and  arrests  soneone  for  being  unsafe.  That  may  be  the  case  in  certain  types 
of  operations,  but  this  is  not  the  case  in  system  safety.  To  change  designs 
or  operations  costs  time  and  money.  Changes  are  very  expensive.  For 
example,  it  costs  $25,000  to  put  a  simple  decal  on  a  fleet  of  F-4  aircraft. 
Complicated  design  hardware  changes,  of  course,  will  be  much  more  costly. 

In  order  to  change  the  design  of  the  system,  you  must  have  legitimate 
cl  dims,  backed  up  by  clear  logic  which  can  convince  the  program  manager  that 
any  hanges  are  warranted.  He  is  concerned  with  cost,  schedule,  and 
performance.  If  you  can  show  where  the  added  safety  will  Improve  the 
performance  of  the  system  to  a  point  where  it  is  cost  beneficial  to  do  so. 


then  you  have  a  good  chance  to  oonvlnce  him.  Without  that  kind  of  logic, 
you  will  not  get  the  changes  that  you  recoomend.  These  notes  will  expose 
you  to  some  of  the  logic  which  you  can  measure  performance  and  Justify 
changes  to  hazardous  operations  or  designs. 

The  systems  safety  process  is  a  requiroaent  levied  upon  all  services  by 
Department  of  Defense  Instruction  5000.36.  That  DODl  is  Implemented  in  the 
Air  Force  by  AFR  80(^16.  It  is  Implemented  in  Systems  Ccmmand  through  AFSC 
AFFTC  supplements  to  AFR  127-2.  All  the  Services  have  adopted  Mil  Standard 
882B,  which  defines  the  systems  safety  process  to  be  lavlemented  by 
contractors  when  developing  and  designing  new  systems.  There  are  two  other 
additional  source  docunents  dealing  with  systems  safety:  Systems  Command 
Design  Handbook  1-6,  and  our  local  Air  Force  Flight  Test  Center  Regulation 
127-3.  All  Implement  systems  safety  process  on  us  here  at  Edwards  AFB. 

The  Program  Manager,  or  the  SPO,  buys  a  system  safety  program  fl'om  the 
contractor  similar  to  going  to  the  supermarket  and  buying  a  can  of  beans. 
Basically  these  regulations  direct  us  to  buy  that  can  of  beans,  the  system 
safety  process.  The  program  manager  has  four  different  program  levels  of 
system  safety  that  he  can  buy.  A  docunmnt  called  a  data  item  description 
(DID)  describes  this  "can  of  beans"  that  you  buy  lY*am  the  contractor.  You 
order  that  data  on  a  particular  form  called  a  OD  Form  1664.  That  product 
then  becomes  a  part  of  the  contractor  data  requirements  list  (CUIL).  There 
are  four  DID  dealing  with  system  safety  (see  Attachment  III).  First  of  all 
Is  DI-H-7047,  which  is  the  Sysivns  Safety  Prograa  Plan.  This  DID  describes 
who  is  doing  what,  to  whom,  and  when  they  are  to  do  it.  The  next  did  Is  a 


DI-H-7048,  Systems  Safety  Hazard  Analysis  Report  which  defines  the  type  of 
analyses  the  contractor  will  perform  and  v^n  these  will  be  delivered. 

These  two  products  are  generally  required  for  major  test  programs,  say  for 
exanple  the  design  of  an  F-19,  or  F-20,  or  B-3,  should  those  come  to  pass. 
For  lesser  programs,  you  can  buy  a  smaller  can  of  beans.  DI-H-7049,  the 
Safety  Assessment  Report,  Is  often  used  for  small  development  programs,  and 
finally,  DI-H-7050,  System  Safety  Engineering  Report,  Is  used  for  periodic 
reviews  or  for  minor  changes  like  ECPs,  Class  II  modifications,  etc. 

When  designing  systems,  there  is  an  obvious  system  safety  {decadence. 
First  you  should  design  to  eliminate  hazards  entirely  and  make  the  system 
Murphy-proof.  Obviously  this  cannot  always  be  done,  so  your  next  choice  Is 
to  provide  safety  devices  to  minimize  the  hazard.  For  example,  relief 
valves  on  pressure  tanks  nr  fx-otective  devices  such  as  clothing  worn  by 
individuals  working  with  caustic  material.  If  the  problan  cannot  be 
e  iminated  or  a  suitable  safety  device  cannot  be  designed,  then  you  should 
provide  warning  devices.  You  are  familiar  with  the  fire  detection  systems, 
pedal  shakers,  warning  horns,  etc.,  installed  on  aircraft.  And  finally,  the 
last  of  the  safety  priorities  is  a  procedures  change.  A  change  can  be 
mir.imized  by  the  introduction  of  a  Note,  a  Warning  or  Caution  in  the  Flight 
Manual.  All  too  often  this  is  taken  as  the  easy  way  out.  Notes  and  changes 
in  r light  Manuals  come  cheap,  and  it  is  very  tempting  to  use  these  as  the 
corrective  action  for  a  major  design  fault.  But  what  help  is  it  to  the  F-4 
pilot  to  put  6  note  in  the  flight  manual  saying  "WARNING,  YOU  MAY  LOSE  YOUR 
TACAN  WHEN  YOU  PUT  YOUR  GEAR  DOWN."  That  doesn't  do  him  much  good  when  he's 
in  the  soup,  groping  frexn  the  final  approach  fix  to  the  touchdown  point 
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without  a  GCA.  Obviously  preoadanca  dapends  on  how  much  money  you  hava  and 
how  savera  the  hazard  is.  That  is  the  heart  of  the  system  safety  process; 
defining  hazards,  defining  risk  and  calculating  what  to  pay  for  oorractlons. 
The  whole  overaU  system  safety  process  is  somewhat  mialcgous  to  forecasting 

the  future:  trying  to  identify  hazards,  and  trying  to  eliminate  them  tram 
the  system. 

The  use  of  Lessons  Learned  to  uiderstand  the  mishaps  end  mistakes  of  the 
past  are  invaluable  in  predicting  what  will  hq>pen  In  the  future. 

Hlstprlcal  data,  available  to  you  as  the  design  engineer,  the  test  engineer, 
or  the  test  pilot  on  a  particular  program,  include  the  following: 

a.  First  of  all  Is  the  Air  Force  AFSC  Design  Handbook  series  which  has 
a  good  catalog  of  designs  and  lists  of  various  standards,  codes  and 
specifications  for  various  systems,  not  only  aircraft,  but  physical  plants 
as  well. 

b.  The  military,  government,  or  professional  codes  and  standards.  For 
example  the  electricity  in  buildings  is  covered  by  electrical  codes,  fire 
protection  codes,  etc.  These  are  very  important  to  the  design  engineer  to 
minimize  the  potential  for  mishap  or  to  reduce  the  nature  of  any 
cata8trc.;AM,  in  case  one  does  occur. 

c.  There  are  also  ocmprehenslve  prograns  in  lessons  learned  throughout 
the  Air  Force.  Air  Force  Systems  Ccmmand  Regulation  800-10  obliges  each  of 
the  major  test  centers  to  prepare  a  lessons  learned  ainually.  These 
documents  are  available  at  the  library  and  lessons  am  be  implied  when 


planning  and  conducting  flight  test  programs. 


d.  In  addition,  Air  Force  Acquisition  Logistics  Division,  a  major 
division  of  AFLC,  has  a  retrieval  system  through  which  lessons  learned  can 
be  derived  from  field  reports  sent  by  the  various  users.  Air  Logistics 
Centers,  and  item  and  system  managers  throughout  the  AFLC  system. 

e.  Don't  forget  the  Air  Force  Inspection  and  Safety  Center  at  Norton 
which  has  a  huge  data  base  on  all  sorts  of  mishaps  and  causes.  This 
information  is  available  to  you  as  a  test  program  manager  and  designer.  All 
that  is  required  is  to  call  to  Norton  and  they  will  be  glad  to  oblige  and 
will  assist  you  by  providing  existing  data  or  by  even  designing  a  separate 
data  tailored  to  your  needs. 

f.  Throughout  the  Center  there  are  all  sorts  of  reports  on  past 
programs.  These  reports  tell  not  only  how  the  program  was  structured,  and 
how  it  was  conducted,  but  in  all  the  reports  since  1978  the  safety  planning 
data  is  also  included.  Safety  planning  data  not  in  these  test  reports  is 
available  in  the  System  Safety  Office  or  the  Tla.ch  Library.  These  data  are 
known  as  the  AF  System  Command  Form  5028  and  deals  with  the  safety  and 
test  planning  to  conduct  the  test.  At  the  end  of  each  test  program  a  final 
closeout  memo  for  record  is  prepared  discussing  the  adequacy  of  safety  planning. 
"Was  it  too  restrictive;  did  it  overlook  hazards  that  occurred  or  were  there 
any  unusual  occurrences."  These  forms  are  on  file  at  the  System  Safety  Office 
and  information  such  as,  test  type,  aircraft  type,  key  test  words,  and  test 
process  changes  are  maintained  on  a  computer  data  base  for  easy  access  by  all 
project  personnel. 

g.  Talk  to  the  old  heads.  There  are  numerous  folks  at  the  Center  and 
throughout  Systems  Command  who  have  conducted  tests  for  years  on  a  number  of 


systems.  The  use  of  that  corporate  memory  la  invaluable  In  conducting  test 
programs. 

The  primary  thrust  of  the  system  safety  process  is  to  determine  the 
hazards  associated  with  the  particular  design,  plan,  or  concept,  and  to  make 
value  Judgments  as  to  the  acceptability  of  the  risk  Involved.  These 
processes  are  know  as  hazard  analysis  and  risk  management  respectively. 
There  are  three  goierally  accepted  schemes  of  determining  and  evaluating 
hazards:  the  first  Involves  Inductive  logic,  often  refwred.  to  as  "fhcm  the 
bottom  to  the  top,"  which  generates  a  specific  situation  to  an  overall 
effect.  Secondly,  deductive  logic,  "fy*om  the  top  down,"  reasons  from  a 
general  result  down  to  the  specific  cause;  and  finally,  Intuitive  logic  is 
another  way  of  saying  "experience." 

Let's  begin  with  inductive  type  logic,  "bottom  to  the  top  type 
analysis."  Consider  the  electrical  systan  on  your  automobile.  Look  at  the 
specific  fault  inodes  in  the  voltage  regulator.  If  the  points  in  your 
regulator  freeze  closed,  what  would  be  the  overall  effect  on  your  car's 
operation?  Obviously  if  they  are  frozen  closed  the  output  current  from  the 
alternator  will  be  excessive  and  will  eventually  boil  yotr  battery  dry  or 
0vm  ofiuse  it  to  explode.  Eventually  you  will  be  imable  to  make  future 
starts  although  you  can  probably  continue  to  rim  for  quite  some  time  with 
the  points  frozen  together.  On  the  other  hand,  if  the  points  were  stuck 
apart  so  no  excitation  current  is  provided  to  yovr  alternator,  and  there 
would  be  no  output  and  the  battery  would  eventually  discharge.  The  distance 
you  could  travel  would  be  dependent  on  the  rate  of  electrical  drain  on  the 
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battery  but  eventually  the  automobile  will  no  longer  run.  Such  analysis  is 
called  "failure  mode  and  effective  analysis."  Starting  with  the  bottan, 
specific  "black  box"  or  some  of  the  simple  parts  of  that  black  box,  failures 
are  generalized  effect  to  the  overall  system.  This  is  "bottom  to  the  top" 
or  inductive  type  reasoning. 

Now  consider  the  top  down  or  deductive  type  reasoning.  The  general 
event  is  car  failure  to  run.  Why  would  your  car  cease  to  run?  It  could  be 
stopped  for  a  number  of  reasons  but  let's  start  very  simply.  For  exanple, 
if  your  car  w:.ll  not  start  In  the  morning,  it  could  be  one  of  two  common 
reasons.  Oie  -  you  don't  have  sufficient  fVjel  to  the  engine,  or  secondly, 
you  don't  have  the  proper  ignition.  There  are  several  ways  you  may  not  have 
proper  fuel:  you  may  be  out  of  fuel;  maybe  the  fuel  mixture  is  too  rich;  or 
maybe  the  fuel  mixture  is  too  lean.  On  the  other  hand,  why  would  you  not 
have  spark?  Maybe  there  is  a  complete  absence  of  spark.  Maybe  the  spark  is 
Uw  weak  or  maybe  the  spark  is  occurring  at  the  wror«  time.  You  could 
furtner  analyze  vAy  each  of  those  events  did  not  occur  and  would  finally 
arrive  at  some  probable  causes  through  logic  fhom  the  top  down.  That 
particular  type  of  logic  is  most  commonly  used  in  system  safety  hazard 

analysis  and  the  particular  method  in  which  it  is  used  is  called  fault  tree 
analysis. 

Fault  tree  logic  uses  a  convention  which  is  shown  in  figure  1. 
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INFORMATIUN  Ok  SIGNIflCAHCE  TtliNG  BfLOW  TO  ANOTHER  AREA  lUENIl 

t  itu  .-.y  THE  triangle  with  a  line 

- — - - - prawn  i  ROM  THE  APEX. _ _ 


Consider  the  simple  system  shown  in  figure  two.  It  is  a  typical  air 
compression  that  you  might  find  in  a  filling  station;  an  electrical  naotor 
powers  an  air  pump.  When  the  air  reaches  sufficient  pressure  a 
pre3sure-acti\rated  swltdi  shuts  off  electrical  power,  terminating  pump 
operation.  Consider  a  hazard  of  the  tank  exploding:  a  simple  fault  tree 
analysis  would  look  as  shown  in  figure  three. 

What  importance  is  this  to  you  as  the  design  engineer  or  the  test 
manager?  A  fully  developed  fault  tree  may  cover  enough  space  to  paper  a 
whole  wall  the  size  of  the  classroom  in  the  Test  Pilot  School. 

Consider  the  simple  analysis  shown  in  figure  four.  An  undesired  event 
could  occur  as  a  result  of  a  combination  of  any  four  different  events  -  a, 
b,  c,  or  d.  The  probability,  shown  to  the  side  of  the"a,"  "b,"  "c,"  or  "d" 
is  the  likelihood  that  those  would  occur.  If  you  were  the  design  engineer 
and  had  a  limited  budget  to  reduce  the  probability  of  occurrences  of  any  of 
those  four  events,  to  which  would  you  apply  your  money?  At  first  glance  it 
would  seem  most  obvious  that  you  should  reduce  "d"  since  that  is  by  far  the 
most  likely  of  the  events  to  occur.  However,  if  you  consider  the  fault  tree 
you  will  see  that  the  overall  undesired  event  will  occur  anytime  "a"  happens 
by  itself.  Hiat  is  called  a  single  point  failure.  The  probability  of  the 
overall  undeslred  event  is  .005,  the  same  as  "a."  Only  one  other  unique 
combination  oi'  events  will  cause  the  overall  undeslred  event:  "b"  and  "c” 
and  "d"  all  occurring  simultaneously.  The  probability  that  all  three  occur 
simultaneously  is  "a”  x  "b”  x  "c"  or  .000006,  a  very  tmllkely  event.  Of 
course,  you  see  that  "b"  and  "a"  could  cause  the  event  or  "a"  and  "b"  and 
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”0  could  coour  -  theso  are  houever  trivial  alnoe  "a”  will  have  triggered 
the  event  already.  Very  coaiijllcated  ayatema  can  be  analyzed  quickly,  by 
computer  proceaaea  now  available  to  aafety  dealgn  englneera,  to  identlly 
alngl.  point  fallurea,  dual  point  fallurea.  and  three  point  fallurea,  etc., 

on  very  ccnplex  ayateaa  fault  wee  analyala  can  Identify  ubere  the  hazard,’ 
are  most  likely  to  occur. 


Intuitive  logic,  or  experience,  can  be  brought  to  use  in  a  variety  of 
manners.  Oie  way  is  the  use  of  the  hazard  analysis  similar  to  that  we  use 
in  theAF  Systems  Command  Form  5028;  there  are  many  other  variants  of  this 
sort  of  document  throughout  industry  and  some  are  shown  In  figures  5-7  . 
Generally  they  follow  the  format:  hazard,  cause,  effect,  hazard  category 

(in  terms  of  MII^PEC,  882B),  and  finally,  corrective  action  that  will 
minimize  or  reduce  these  hazards. 


Another  very  powerful  technique  in  identifying  unique  hazards  is  called  a 
sneak  circuit  analysis.  Consider  figure  8.  Ihls  is  a  portion  of  the 
typical  elect -ical  system  in  an  automobile.  The  radio  is  normally  played  O) 
turning  on  the  ignition  switch  and  the  radio.  Note  that  with  the  ignition 


.witch  on  and  pressing  on  the  brake  pedal  lights  up  the  tail  lights.  The 
...ergency  fla.her  switch  also  lights  up  the  tall  lights.  Now,  consider 
this:  Your  ignition  switch  is  off.  (Normally  you  would  not  be  able  to 
operate  the  r.«lio.)  You  turn  on  the  emergency  flasher  and  press  on  the 
brake  pedal.  Power  would  now  be  applied  to  operate  the  raaio 
intermittentlvl  That  is  know,  as  a  sneak  circuit.  While  of  sane  casual 
interest  here.  It  is  no  hazard.  However,  if  this  were  not  a  radio  but  a 
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pyrotechnic  device,  the  result  of  a  sOeak  circuit  could  be  catastrophic. 

Sneak  circuits  can  sonetimes  be  of  value.  NASA  discovered  that  an 
experiment  in  a  spacecraft  oould  be  initiated  thrcu^  a  sneak  circuit,  NASA 
considered  it  not  to  be  a  safety  hazard  and  the  vdrir»g  retained  unchanged. 
When  the  space  craft  was  in  orbit  this  particular  experiment  did  not 
function  vhen  the  switch  was  turned  on,  NASA  was  able  to  operate  the 
experiment  through  the  use  of  the  sneak  circuit. 

Let  us  examine  risk  assesanent.  The  risk  is  nothing  more  than  the 
product  of  the  severity  of  a  inishe?>,  should  it  occur,  and  the  frequency,  in 
vhich  it  will  occur,  or  alternately,  is  severity,  the  rate  and  the  esqxssure 
factor:  R  =  Sxf  =  SxRxt, 

Consider  for  a  manent  this  sanplej  If  someone  were  to  borrow  ny  pickup 
truck,  I  oould  evaluate  the  risk  in  actual  terms.  The  cost  of  the  pickup 
frcm  the  Red  Book,  might  be  $5,000,  To  determine  the  accident  rate  I  look 
at  Air  Force  accident  records  for  the  past  several  years  to  find  that  the 
accident  rate  is  approximately  three  accidents  per  million  miles.  My 
neighbor  wanted  to  borrow  the  vehicle  to  drive  200  miles;  the  risk,  to  me, 
would  be  shown  hy  the  following  equation*  3.00  =  $5,000  x  3/1,000,000  x  200 
the  risk  it  involves  about  $3.00  or  a  cent  and  a  half  per  mile,  assuming  I 
have  no  ccnprehensive  insurance.  That  would  be  a  method  of  evaluating  an 
insurance  policy,  if  I  can  bty  insurance  for  less  than  a  cent  and  a  half  per 
mile  it  would  be  in  ny  benefit  to  buy  that  insurance.  If  it  were  nore  than 
that,  then  I  would  consider  a.'lf-insuring.  Safety  devices  aid  systan  safety 
planning  can  be  considered  similarly.  Unfortunately,  in  the  real  world 
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predicting  actual  rates  and  actual  exposures  is  not  precise.  Exact  physical 
probabilities  are  usually  anavailable  and  you  must  rely  on  subjective  type 
analysis.  Three  subjective  type  analysis  will  be  p-esented. 

First  of  these  methods  is  known  as  the  Risk  Assessment  Code  (RAC). 
Consider  figure  9.  The  hazard  levels,  defined  by  severity,  are  broken  into 
four  broad  categories  ranging  iV-an  catastrophic  to  negligible.  (These 
Categories  I,  II,  III,  and  IV,  are  defined  in  MIL  STANDARD  882  BJ  The 
likelihood  tht;  hazard  will  occur  is  identified  alphabetically  "a"  through 
"d."  Something  likely  to  occur  within  a  short  period  of  time  is  assigned 
probability  "ct,”  something  unlikely  as  to  occur  is  {(ssigned  probability  "d." 
The  severity  and  likelihood  levels  are  combined  into  a  matrix  as  shown  in 
figure  9c.  TTis  method  is  often  used  when  a  hazard  is  Identified  on  a 
hazard  report  submitted  to  the  Flight  Test  Center.  The  Arabic  numbers  in 
the  matrix  1  through  6  assist  the  manage*  or  the  coomander  in  prioritizing 
use  of  his  limited  funds  or  personnel  to  correct  hazards  that  are  reported 
to  him.  He  will  start  with  the  lowest  numbers  and  apply  his  time  and  energy 
to  solving  those,  leaving  the  5s  and  6s  to  be  fixed  last. 

In  our  system  safety  planning  at  the  Flight  Test  Center  we  use  a 
procedure  similar  to  the  risk  assessment  code.  Using  the  same  hazard 
categories  as  defined  by  MIL  STANDARD  882  B,  we  define  three  levels  of  hazard 
probabilities;  high,  medium  or  low  (see  figure  10).  The  slanted  lines 
define  three  levels  of  risk:  The  upper  left  hand  corner  equates  to 
Hazardous  Tests.  (Category  I  or  II  and  high  probability  of  occurrence. 

During  a  hazardous  test  the  Center  Commander  must  be  briefed  prior  to  each 
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flight.)  The  next  area  is  Medium  Risk.  (Cat  I  or  II  with  moderate 

of  oocijf  cfiofc  or  Cat  III  with  the  high  probability  of  occurence.) 
These  tests  must  be  briefed  to  the  wing  commander  prior  to  flight.  Finally, 
those  tests  that  fall  in  the  lower  right  hand  corner  are  considerd  Low  Risk 

and  are  managed  by  the  squadron  oommander  at  or  by  the  combined  test  force 
commander. 

Another  subjective  method  of  quantifying  risk  is  the  use  of  the  Real 
Hazard  Index  (RHI).  Although  similar  to  the  risk  assessment  code,  it  does 
not  use  the  matrix  (see  figures  11  and  12).  The  severity  of  hazard  severity 
is  defined  in  MIL  STANDARD  882B^  and  each  category  is  assi©ied  a  risk  value 
of  one  through  four,  shown  on  the  right  hand  side  of  the  figure  11.  Figure 
y  depicts  word  descriptors  for  the  probability  of  occurrence  of  those 
hazards.  Now  the  Real  Hazard  Index  is  nothing  more  than  the  product  of 
those  two  numbers:  ranging  fVom  a  law  of  one  to  the  maxlmixn  of  2^.  The  SPO 
or  the  program  manager  will  often  make  an  arbitrary  decision  to  correct  any 
hazard  with  a  real  hazard  index  of  greater  than  some  preselected  value,  say 
12,  and  ignore  those  below.  Attached  to  these  notes  you  will  find  a 
document  called  a  Risk  Management  Guide  for  Air  Force  Operations  published 
by  the  C  .rectorate  of  Safety  at  Norton  AFB.  In  this  text  you  will  also  find 
another  description  of  the  real  hazard  index  using  different  severity  levels 
and  an  entirely  different  set  of  word  descriptions.  The  Real  Hazard  Index 

can  be  used  with  many  types  of  hazard  severity  definition  or  probability 
description. 

Another  method  of  quantifying  risks  was  recently  developed  by  two 
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gentlemen  fVom  the  Navy:  Mr  Kenneth  J.  Graham,  a  research  chemist  at  the 
Detonation  Physics  Division  of  the  Naval  Weapons  Center  at  China  Lake,  and 
Dr  Gilbert  H.  Kennedy,  a  distinguished  professor  iVom  Chemical  Engineering 
at  the  Naval  Postgraduate  School  at  Monterey,  California.  Together  they 
conducted  research  on  accident  statistics  and  risk  analysis  and  then 

published  an  article  called  a  Practical  Safety  Analysis  System  for  Hazards 
Control. 

Their  analysis  is  based  on  the  following:  risks  are  the  product  of  the 
severity,  the  likelihood,  and  the  exposure  factor  (Risk  =  Severity  x 
likelihood  x  exposure).  The  severity,  or  consequence  as  they  call  it,  is 
shown  in  figure  13.  The  consequence,  ranging  fVom  1  to  100  as  they  call  it, 
i:>  determined  subjective  by  word  description  or  may  be  determined 
empirically  by  the  following  equation:  C  s  ($damage/100)0.4.  The 
iikellnood  is  alst;  defined  by  a  continuum  from  .1  to  10:  from  something 
that  is  virtually  impossible  to  something  that  might  well  be  expected.  See 
figure  1^.  The  exposure  factors  are  based  on  the  word  descriptions  shown  in 
figure  15.  A  product  of  those  three  numbers  yields  a  score  used  to 
subjectively  evaluate  the  risk.  See  figure  16.  This  graduation  of  risk  is 
more  useful  than  the  Real  Hazard  Index. 

Practical  use  of  this  method  might  be  as  follows:  For  some  hazards  that 
have  a  very  high  risk  (above  320)  that  point  could  be  a  point  where  you 
might  consider  grounding  the  fleet  or  grounding  the  airplane.  It  is  a  high 
risk  with  immediate  correction  required  (160-320),  one  might  opt  for 
grounding  the  airplane  within  ten  days  pending  some  sort  of  an  inspection. 


Z. 


NOTICEABLE,  FIRST  AID  flAYBE  $10^-10 


LIKELIHOOD 


3.33 


EXPOSURE  factors 


•FIGURE  15 


'5.3  W 


TIMES  PER  YEAR 


flGURE  16 


■i 


correction  is  required  (70-160)  one  might  opt  for  a  routine  type  TCTO; 
there  is  no  great  urgency  but  the  situation  should  be  corrected.  If  the 
risk  score  is  20  to  70,  attention  is  needed,  but  that  might  be  the  type  of 

action  where  only  a  Note  or  a  Caution  in  the  dash  one  is  warranted  to 
correct  the  action. 

Further,  these  two  gentlemen  proposed  a  method  to  evaluate  the  cost 
effectiveness  of  any  corrective  action  that  might  be  proposed,  in  this 
analysis,  the  risk  reduction  effectiveness  is  the  product  of  a  risk 
reduction  multiplier  and  risk  score  previously  developed,  divided  by  a  cost 
divisor.  The  risk  reduction  multiplier  is  the  percentage  of  the  reduction 
of  the  hazard.  For  example,  something  that  would  reduce  the  hazard 
potential  by  60X  is  assigned  a  risk  reduction  multiplier  of  .6;  if  it  is 
100%  risk  reduction,  then  the  reduction  value  is  1.0.)  The  cost  divisor  is 
e-plrlcally  derived  coat  (V<»,  (Cd  =  Speelfloally.  It  la 

the  cube  root  of  a  total  cost  divided  by  100.  The  cost  effectiveness  is 
determined  by  the  equation:  cost  effectiveness  =  Ktaiicn(,ri  X 

The  degree  of  effectiveness  of  various  systems  can  be  evaluated  to  this 
manner.  (See  figure  17).  Effectiveness  scores  greater  than  20  indicate 

that  the  corrective  device  is  very  worth  while;  scores  less  than  10  Indicate 
the  fix  is  of  doubtful  merit. 

The  matheuatlca  uaed  In  ayatema  aafety,  are  nothing  more  than 
probabUltlea  and  atatlatloa.  .<  abort  refl-eaher  In  probabllltlea  la  In 
order.  Conald«-  the  probability  that  during  the  flip  of  a  coin  It  will  be  a 
head.  Ihla  la  obvloualy  50-50,  aaaunlng  It  cannot  land  on  the  edge.  We 
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reach  that  conclusiot^  a  priori^  by  simply  deducing  the  possible  ways  it  can 
land  each  flip  divided  by  the  total  number  of  ways  it  can  land.  Similarly 
the  probability  of  any  given  nunber  on  a  die  will  appear  is  1/6,  only  one 
number  can  come  up  although  there  are  six  possibilities. 

Consider  the  probability  of  being  able  to  roll  five  6a  in  one  roll  and 
win  a  free  drink  at  the  bar.  That  is  the  chance  of  the  first  dice  being  a 
six,  the  second  one  being  a  six,  the  third  one  being  a  six  and  so  on.  The 
total  probability  that  you  can  roll  five  6s  in  one  roll  will  of  course  be 
1/6  times  itself  five  times.  (Probability  =  (1/6)5).  That  means  the  first 
dice  is  a  six  and  the  second  one  is  a  six  and  the  third  one  is  a  six,  etc. 
When  you  use  the  verb  "and”  you  are  multiplying  individual  probabilities. 

Consider  another  possibility:  what  is  the  probability  of  five  sixes  in 
one  roll  or  five  aces  in  one  roll,  in  which  case  the  bar  will  buy  you  and 
your  friends  a  drink.  The  probability  that  you  may  roll  five  sixes  in  one 
roll  or  five  ones  in  one  roll  are  simply  the  two  separate  cases  added 
together.  Note  the  verb  "or"  means  to  add.  That's  the  probability  of  five 
sixes  in  one  plus  the  probability  of  five  ones  in  one I  Those  are  very 
important  in  using  the  "and"  and  "or"  gates  in  the  fault  tree  analysis  v^ich 
was  discussed  earlier.  When  you  reach  an  "and"  gate,  the  two  probabilities 
are  multiplied  together;  when  it's  an  "or"  gate  it's  the  sun  of  all  the 
probabilities  going  into  that  "or"  gate. 

Consider  for  a  moment  the  popular  Soviet  party  game  called  Russian 
Roulette.  At  first  glance  you  will  recognize  the  probability  of  becoming  a 
fatality  in  a  game  of  Russian  Roulette,  if  you  were  to  play  only  one  time. 
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is  obvious  one-sixth:  One  chance  out  of  six.  But  what  are  the  odds  that 
the  bullet  under  the  chamber  is  a  dud  or  what  is  the  probability  the  firing 
pin  might  fail?  Osnsihering  the  >tate  of  mind  of  one  who  would  play  Russian 
Roulette,  what  are  the  chances  that  he  actually  would  hit  himself  when  he 
pulled  the  trigger?  You  can  see  that  actually  the  odds  of  death  would  be 
the  product  of  all:  (1/6  and  probability  of  no  dud  ^  probabllty  of  no 
failure  and  probability,  of  not  missing.) 

How  might  you  determine  the  various  reliability  rates?  Determining  the 
bullet  reliabilities  are  probably  rather  easily  obtained.  You  can  locate 
the  manufacturer  who  has  a  guaranteed  a  specified  rate  of  reliability  on  his 
rullets,  say  no  more  than  five  out  of  a  thousand  would  fall.  The 
reliability  is  at  least  .995.  O  perhaps  he  may  have  the  acceptance  test  on 
that  particular  lot  of  amnunition  where,  out  of  a  thousand  rounds  fired  at 
random  out  of  the  lot  only  four  failed;  therefore,  the  reliability  of  that 
(articular  lot  o'*  amnunition  could  be  considered  .996.  These  are 
■'sripirioali  derived  reliability  figures.  Calculating  the  reliability  of  the 
revolver  might  be  a  little  more  difficult.  Each  manufacturer  generally  has 
<(ji  acceptance  test  which  his  product  mtist  pass  before  it  is  delivered  to  the 
marKet.  Every  gun  is  test  fired.  During  the  initial  firings  revolvers  have 
a  high  failure  rate,  say  perhaps  20  out  of  a  thousand  fail  to  fire  the  first 
time.  The  revolver  that  fails  to  fire  is  returned  to  the  factory  production 
divi-iou  for  refurbishment  eg.  to  replace  a  broken  spring,  dent  hammer,  etc. 
whiU.  or  il  v.'ci3  that  caused  the  failure  to  fire.  That  revolver  is  again 
returned  for  acceptance  test  and  on  the  second  attempt  to  fire,  generally  it 
would  fire.  By  the  time  each  pistol  is  fired  ten  times  the  reliability  rate 


is  very,  very  high,  say  maybe  one  out  of  a  thousand  would  fail  to  fire.  If 
a  sample  of,  say  ten  or  more  revolvers,  were  tested  to  complete  failure,  or 
to  wear  out,  you  would  see  a  curve  that  looks  something  like  that  shown  in 
figure  18.  As  you  near  the  service  life  of  the  revolver,  the  springs  begin 
to  break,  the  firing  pins  wear,  etc.,  as  the  revolver  wears  out.  This  curve 
Is  called  a  "bathtub  curve"  because  it's  shaped  somewhat  like  a  bathtub,  and 
it  is  used  for  predicting  reliability  of  hardware.  A  word  of  caution:  when 
you  are  doing  flight  teat  you  are  actually  performing  on  the  left  hand  side 
of  the  bathtub,  so  the  failure  rates  generally  are  a  lot  higher  than  those 
of  a  proven  system! 

Now  let's  suppose  for  a  moment  that  the  reliability  of  the  revolver  and 
the  bullets  are  of  sufficiently  high  order,  that  you  can  consider  the 
probabilty  being  killed  in  a  round  of  Russian  Roulette  to  be  1/6. 

If  six  people  were  going  to  play  the  game,  and  after  each  player  pulled 
the  trigger,  should  he  survive,  he  would  spin  the  chamber  and  hand  it  to  the 
next  guy.  If  3  player  was  fatally  injured,  the  remaining  players  would 
reload  and  continue  the  game  intil  all  six  had  the  opportunity  to  play. 

What  is  the  probability  that  six  can  play  and  someone  be  killed  in  the  game? 
That  is  the  probability  that  one  gets  killed,  plus  the  probability  that  two 
would  be  killed,  plus  the  probability  that  three  would  get  killed  etc.,  up 
to  the  probability  that  all  six  were  fatally  injured.  Since  the  sum  of  all 
Individual  probabilities  must  etjual  one,  you  could  also  solve  that  problem 
by  subtracting  the  probability  that  none  were  killed  from  one  or  ijiity: 

1  =  Pq  +  Pi  +  P2  ♦  ...  Pn  P  0  =  1  -  Pq*  Now  what  is  the  probability  that 
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none  are  killed?  Well  that's  the  probability  that  a  player  would  survive  on 
any  one  round,  and  the  next  guy  would  survive,  and  the  third  guy  would 
survive,  etc.  or,  in  other  words,  one  minus  5/6  raised  to  the  sixth  power, 
and  that  turns  out  to  be  (5/6 or  .33^9.  The  probability  that  someone  is 
killed  is  then  .6651. 

Now  if  you  were  to  ask  an  alternate  question,  nanely  what  is  the 
probability  that  exactly  two  were  killed  and  exactly  four  survived,  how 
would  you  solve  that?  How  many  combinations  are  there  of  two  becoming 

killed  and  four  surviving?  The  first  two  could  be  killed  and  the  last  four 

survive,  the  first  and  the  last  one  killed,  and  the  middle  four  survive, 

&! 

etc.  The  number  of  combinations  are  shown  as  follows:  (2!) (41)  s  15, 

n! 

The  general  formula  for  determining  probability  is  shown  below:  xl(-x)! 

p*q  n-K  n  is  the  total  events  in  this  case  six  players;  x  are  the  events  of 

Interest,  l.e.,  2  deaths;  p  is  the  probability  that  death  will  occur,  in 

this  event  one  sixth;  q  is  the  probability  that  death  will  not  occur,  in 
this  event,  five  sixths.  If  you  solve  this  you  find  the  probability  of 
exactly  two  killed  and  four  survive  is  .16745. 

Sc’dom  do  you  have  the  opportunity  to  work  with  specifics  such  as 
described  above.  Generally  you  must  consider  a  continuun  of  events,  or 
continuum  of  probabilities.  For  that  you  use  a  general  fomula  for 
empirical  distribution  called  "a  Poisson  distribution."  Here  is  the 
formula: 

I .  c  *}Ut  ^  (lit)  e. 

\  r  33  - 

In  the  world  of  accident 

prevention  you  are  interested  in  preventing  the  first  accident.  What  is  the 
probability  of  all  possible  number  of  accidents  occurring:  one,  two,  three, 
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Etc?  As  befoi'e,  solve  by  subtracting  probability  of  no  accidents  occurring:  , 
L'cra  1.  ^  d  -  i  -  e  /  -  /e  .  This  is  Murphy's  Law;  it  shows  that  if 

l  I  is  anything  other  than  zero,  (if  there's  a  way  for  sornbody  to  snrvw  I' 
Mpt  1 1' III  it  llict'c  ifi  enough  time,  ( noiiiftxxly  wl]  I  in  fact  nrrew  It  up)  and  tti« 
prol ability  Pany  approaches  unity  (accident  will  occur). 

Limbda  is  the  tailure  rate  or  the  inverse  of  the  mean  time  between 
fail  u'e  (MTBF).  If  so  you’re  runing  a  test  rig  to  determine  mean  time 
bt: tween  failure,  yo\i  can  extract  the  failure  rate  directly, 

Fvst-cjn;^  Safety,  as  applied  here  at  the  Flight  Test  Center,  is  more 
i-rt  I  lately  call»=^ci  Test  Program  Safety,  since  we  are  here  more  interested 
;i<  .jfiu.'!  (•()ni]t.'’i  iiu-  test  than  wc^  are  of  the  phyr.io.il  design  of  the 
fkuvJwire.  'u  i,  as  te-t  -Migineer  or  the  flight  test  pilot,  prepare  a  test 
,1  iTi.  WfiLle  you  ai'»  i 'rf'parung  d  test  plan,  you  must  consider  the  kazardr; 
inv'olv^  >i.  MsiinJ  the  AFSC  Form  5028  described  earlier  in  the  text,  list  all 
I  lie  .  .irdFy  their  causes,  their  effects,  and  controls  you  might  employ  t^> 
prv'Vf  I:  the  occurrence  oi  that  hazard. 

V..:'  n  yi/ui’  te::i:  plrin  has  been  wiutten,  it  will  be  revi^w^'d  by  engineer^' 
f  u.» 'hnical  oont('nt.  'Iliis  review  is  primarily  technical  although  safety 
■  I  '  •  f  lay  be  considered.  Once  that  review  has  been  accompli  shoo  then  ';o\a 

will  meet:  a  system  odtety  Review  Boai’^i  (SRB).  You,  your  supervisors,  and 
th'-  v'.tem  Safety  nivisiof^  will  muteally  determine  the  canposition  of  the 
iDCDrord.  Tlic  board  will  usaally  consist  of  operation  and  engineering 
representatives  plus  others  as  required:  The  operations  representative  is 
preferably  cuirent  in  the  type  aircraft  under  test  and  is  in  a  supervisory 


or  Independent  position.  The  engineering  representatives  are  supervisors  or 
experts  fVom  the  disciplines  Involved:  i.e.,  propulsion,  aerodynanlc, 
flutter,  etc.  Occasionally  there  will  be  a  maintenance  personnel 
represented  if  there  is  significant  maintenance  involvement.  Many  test, 
need  representation  the  Fire  Department  or  the  hospital  if  lasers  or  toxic 
materials  are  used  or  are  part  of  the  test. 

When  the  deliberations  are  complete  a  risk  assessment  is  determined, 
(hazardous  test,  raediun  risk  or  low  risk  as  shown  by  the  matrix  discussed 
earlier).  The  complete  AFSC  Fom  Form  5028  is  signed  by  board  members  and 
supervisors  as  shown  on  the  ft*ont  side  of  the  Form  5028.  After  the  commanders, 
have  seen  these  docunents,  the  test  plan  and  Form  5028  is  briefed  to  the 
Commander,  Vice  Commander,  and  the  Test  Wing  Connander  regardless  of  the 
risk  level. 


